Analyzing neural network behavior through deep statistical model checking

Abstract Neural networks (NN) are taking over ever more decisions thus far taken by humans, even though verifiable system-level guarantees out of reach. Neither is the verification technology available, nor it understood what a formal, meaningful, extensible, and scalable testbed might look like for such technology. The present paper an attempt to improve on both above aspects. We family formal models that contain basic features automated decision-making contexts which can be extended with further orthogonal features, ultimately encompassing scope autonomous driving. Due possibility model random noise in decision actuation, each instance induces Markov process (MDP) as object. NN this context has duty actuate (near-optimal) decisions. From perspective, externally learnt serves determinizer MDP, result being chain amenable statistical checking. combination MDP encoding action policy central we call “deep checking” (DSMC). While straightforward extension checking, enables gain deep insight into questions “how high NN-induced safety risk?”, good compared optimal policy?” (obtained checking MDP), or “does training NN?”. report implementation DSMC inside Modest Toolset NNs, demonstrating potential various instances family, illustrating its scalability function size well other factors degree training.